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(54> Au^matea ima 9 e cropping using selected compositional rules. 
(57, Animageprocessing^^ 

processing means <6>. and an ''^^^^S^S^ having compositional* significant 
are relevant to the composition of the "^JSsT^ examp ie, considering Figure 2A. the 

properties determined from ^^J^^SSStSlSZ in the image and (b) includesa 
building feature (11) has the propert.es tha : rt ^^wj^ PJJJJ {ormula ted for each of the propert.es; 
relatively high level of detail. Corresponding '^SSXSS^ jma ge feature, and uses these to 
the device operates to select appropnate co mposrtional rules J***""™ ' * na , ing the imag e as grid 

Ke n d S ;magr« 
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Convert captured 

image to YCC 
colour space format 



30 



Blur image 



From new "seed 
areas" that have 
smooth colour 
and intensity 



> 



Grow "seed areas" by 
adding areas with 
the similar colour 
and intensity 



Area 
Growing 




Fig. 




Merge areas separated I 
by weak e dges ) 



I 



Merge areas with 
similar me an colours 



> 



Merge areas enclosed 
by other areas if below 
a threshold size 



|~ToFig.3B \ 



Area 
Merging 



J 



5/10 




Superimpose a grid 
on merged area 
image to define regions 



i 



Average colour and 
intensity within each 
region 



i 
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Compare averaged 
colour and intensity 
within each region 
with that for each 
adjacent averaged 
region 



I 
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Assign an interest 
metric to each region 
depending on the 
difference between that 

region and adjacent 
regions 



-54 



Fig. 



ornt t l r T lnef ° reachre 9 ion °ne 

^^ P °t^ 0na ^ s «cant 
Properties from amongst a 

Plurality of different possible such 

properties, depending on the 
average colour, intensity, location 

•n the captured image, and interest 
metrics within the regions 
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Select compositional rules 
appropriate to the determined 
compositionally significant 
properties 



I 



Determine one or more suitable 
crop boundaries by applying one 
or more of the selected 
compositional rules 



•57 
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(Figs 3A and 3B up to and! 
C including Step 54 J 



Identify features relevant to 
the composition of the electronics 
image using a combination of 
one or more interest metrics 



^460 
I 



|Fig. 3B Steps 55,56,57| 



FORM ONE OR MORE MINIMAL CROPPING RECTANGLES 
(MCRs), TOGETHER WITH CROPPING LIMITS 
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Fig. 4B 



SELECT FIRST MCR AS CURRENT 



J^-62 



IDENTIFY A SET OF POSSIBLE CROP EDGE 
LOCATIONS FOR EACH SIDE 
(EDGE CONSTRAIN TS) 

I 
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64 



IDENTIFY HORIZON LINE IF POSSIBLE ( 1/3 HEIGHT CONSTRAINT) | 



IDENTIFY PRINCIPLE INTEREST POINTS IN EACH AREA OF 
INTEREST IN THE MCR (EACH PT GIVES A 1/3 HEIGHT 
CONSTRAINT AND A 1/3 WIDTH CONSTRAINT) 



65 



Fig. 4B | 



Fig. 4 A 
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^Fig. 3B Steps 55,56,57 j 




PICK NEXT COMBINATION 

OF CONSTRAINTS TO 
DETERMINE FIXED ASPECT 
RATIO CROP CANDIDATE 



PICK NEXT COMBINATION 

OF CONSTRAINTS TO 
DETERMINE UNSPECIFIED 
ASPECT RATIO CROP 
CANDIDATE 




EVALUATE CURRENT CROP CANDIDATE BY: 

A) TESTING EACH EVALUATION RULE TO GENERATE 
A RULE PENALTY SCORE IN RANGE it Li 

_J9 FORMING A WEIGHTED SUM OF THE RULE 
PENALTIES TO GENERATE A TOTAL PENAUTYFOR 
THE CROP CANDIDATE 
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Fig, 4B 
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Fractional distance 
1.0 across Crop candidate 



Fig. 5 




(64CU80) 



Fig. 6 
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j {Figs 3A + 3B 
up to and 
! including 

|Step_54}_ _' 



IDENTIFY FEATURES RELEVANT TO COMPOSITION 



3 
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J 



DETERMINE RELEVANT CONSTRAINT SETS FOR FEATURES -f^g 2 
r I {Rg3B 

93 L 



ENUMERATE THE POSSIBLE I Stef)S 55 > 56 .57} 

ALTERNATIVES IN EACH CONSTRAINT SET I 
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PICK NEXT COMBINATION OF CONSTRAINTS 
ONE ALTERNATIVE FROM EACH CONSTRAINT SET i 



95 



DETERMINE THE OPTIMAL CROP CANDIDATE FOR TUP 
CURRENT CONSTRAINTS. (I.E. THE SOLUTON WTO T^E 
MINIMUM PENALTY SCORE) 



96- 



OPTO^LLY REEVALUATE THE - 
PENALTY FOR THE CROP USING A WIDER 
SET OF EVALUATION METRICS 



97— £ 



RECORD THE CURRENT CROP 
COORDINATES AND ITS PENALTY SCORE 




1 



SORT ALTERNATIVE CROPS IN ORDER 
OF PENALTY SCORE 



3 



102- 



RETURN THE N CROPS WITH THE 
LOWEST PENALTIES TO THE USER 



+ r » * 

m <• * 



10/10 



^Figs 3A + 3B 
up to and 
including 
Step 54} 



IDENTIFY FEATURES RELEVANT TO COMPOSITION 



T-191 



jj>tep 54) — | | 



DETERMINE RELEVANT CONSTRAINT SETS FOR FEATURES 



{Fig 3B Steps 
55,56,57} 

193A- 



ENUMERATE THE POSSIBLE 
ALTERNATIVES IN EACH CONSTRAINT SET 



193B- 



SELECT A GROUP OF CONSTRAINT 
SETS AS DRIVING CONSTRAINT SETS 



194— 

195 
196-v 



197- 



199. 



I 

1 



PICK NEXT COMBINATION OF DRIVING CONSTRAINTS 
ONE FROM EACH DRIVING CONSTRAINT SET 

* 



DETERMINE CURRENT CROP CANDIDATE 



I 



EVALUATE CURRENT CROP CANDIDATE BY: 

A) TESTING EACH EVALUATION RULE TO GENERATE 
A RULE PENALTY SCORE IN RANGE 0..1 

B) FORMING A WEIGHTED SUM OF THE RULE 
PENALTIES TO GENERATE A TOTAL PENALTY FOR 
THE CROP CANDIDATE 



RECORD THE CURRENT CROP 
COORDINATES AND ITS PENALTY SCORE 




201 



SORT ALTERNATIVE CROPS IN ORDER 
OF PENALTY SCORE 



202- 



RETURN THE N CROPS WITH THE 
LOWEST PENALTIES TO THE USER 



Fig. 8 
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Automated Cropping of Electronic Images 

The present invention relates to the automated and 
semi-automated cropping of electronic images, and in 
5 particular to an apparatus and a method of using an 
electronic camera to capture and crop such electronic 
images . 

Conventional photography requires the photographer to use 
10 a viewfinder both to aim the camera and to compose the 
picture. Composition at a particular location is done by 
changing the direction of the camera and altering the zoom 
control. Careful composition takes time and attention, as 
well as an understanding of various rules of good 
15 photographic composition. This is a skill that many find 
hard to learn. The effort required causes the photographer 
to be "out of the event" and in many cases this problem is 
enough to prevent potentially pleasing photographs from 
being taken. This is particularly the case when a 
20 photograph has to be taken quickly, for example when 
photographing action events, or children. Although, in 
principle, a photograph can be cropped after the event, 
this is time consuming and inconvenient, and may still 
require knowledge of the rules of good photographic 
25 composition, which the photographer may not possess. 

It is an object of the present invention to provide a more 
convenient apparatus and method for capturing and cropping 
electronic images. 
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^ Accordingly, the invention provides an electronic image 
processing device for cropping an electronic image, 
comprising an image processing means, the image processing 
means including an electronic processor and firmware 

5 and/or software for processing the electronic image, 
wherein the device is adapted to: 

a) process the electronic image to identify one or more 
features relevant to the composition of the electronic 

10 image, each such feature occupying a sub- region of the 
electronic image; 

b) select one or more compositional rules from a set of 
predetermined, compositional rules, based on the relevance 

15 of the compositional rule(s) to one or more of the 
identified features; and 

c) determine one or more suitable crop boundaries by 
applying one or more of the selected compositional rules. 

20 

Also according to the invention, there is provided a 
method of using an electronic image processing device for 
cropping an electronic image, the image processing device 
comprising an image processing means, the image processing 
25 means including an electronic processor and firmware 
and/or software for processing the electronic image, 
wherein the method comprises the steps of using the image 
processing means to: 

30 i) process the electronic image to identify one or more 
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10 



15 



20 



features relevant to the composition of the electronic 
-age each such feature occupying a sub-region of the 
electronic image; 

ii) select one or more compositional rules from a set of 
Predetermined compositional rules, based on the relevance 
of the compositional rule(s) to one or more of the 
identified features; and 

iii) determine one or more suitable crop boundaries by 
applyxng one or more of the selected compositional rules. 

In a preferred embodiment of the invention, each 

identified feature has one or- „™ • . 

one or more compositionally 

significant properties f rom amongst a plurality of 
different predetermined compositional properties, and one 
or more compositional rules are selected from a set of 
predetermined compositional rules, based on the relevance 
of the compositional rule(s) to the compositionally 
significant properties of one or more of the identified 
features . 



25 



30 



The image my ^ cropped automatically ^ ^ 
processing means according to a measure of the quality of 
the potentia! crop boundaries. Alternatively, the user may 
dually select amongst the potential crop boundaries, so 
that the cropping of the image is performed 
sem a -automatically. i„ this case, one or more of said 
stable crop boundaries is presented to a user of the 
device for manual selection by the user for cropping of 
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the electronic image. 

Compositional ly significant properties of a feature 
include things like the type of feature, e.g. a blank 
5 area, an area of relatively high contrast or colour 
texture, recognisable objects such as a face, the sky or 
an edge. 

The electronic imaging system may be part of an electronic 
10 camera, or a document imaging system, or any other image 
capture system where the captured image may be cropped. 

The electronic imaging system may be used with an 
electronic camera system for capturing an electronic image 
15 of a scene. In particular, the electronic imaging system 
may be incorporated with the electronic camera. 

Alternatively, the system may comprise a conventional 
electronic camera that outputs captured images to an image 
20 processing device, for example a personal computer , that 
includes the image processing means . 

The camera will, in general, comprise a detector array for 
capturing the electronic image and an optical imaging 
25 system arranged to image the scene onto the detector 
array. The camera may be a hand-held still electronic 
camera and/or a video electronic camera. 

The compositional rule may comprise edge placement 
30 criteria, for example having a dark border to an edge of 
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10 



the cropped image, or placing an edge one-third or two- 
thirds the way from an area of interest within the crop 
boundary . 

It may be that just one compositional rule is used 
multiple times on a single image, once for each identified 
feature. 

Once candidate crop boundaries have been determined, these 
may be presented to a user of the system. The user may 
then choose a cropping candidate, whereupon the image 
processing device may be arranged to crop the electronic 
image according to the user's choice. 

15 The identifiable features should be those features that 
are relevant to the placement of cropping boundaries in 
the processed image. 

The camera may include means by which a user of the camera 
20 may tag one or more features relevant to the composition 
of the electronic image, said tagged feature (s) then being 
associated with a compositional rule that includes said 
tagged feature (s) in the cropped image. Such tagging may 
be done by the user indicating by means of suitable 
25 controls a feature or an area of the captured image as 
displayed to the user, for example on a LCD display built 
into the camera. 

It may be possible, however, for features in the image to 
30 be tagged automatically. For example, a person in view of 
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the camera may wear some sort of identifier tag which can 
be recognised automatically by the image processing means 
within the camera. The tag may be an optically 
distinguishable badge pattern understood by the image 
5 processing software. A person can then be automatically 
identified. 

Optionally therefore, the identifiable features may 
include a predetermined feature, for example a tag that a 

10 person may wear. Such a tag may have an identifiable 
pattern to the image processing system recognises. A 
compositional rule will then be associated with such an 
identified tag so that the identified feature is included 
in the cropped image. So that the tag does not distract a 

15 user viewing the cropped image, it is preferable if it is 
implemented as an infra-red only tag (for example, as an 
infra-red transmitter) . 

One way of manually tagging features is to use the camera 
20 viewfinder as a pointing device (as opposed to its 
conventional use as both a pointing and composition 
device) . In this use the prime area of interest is 
deliberately positioned in the approximate centre of the 
frame. When the image is auto-cropped according to this 
25 invention, the region at the centre of the image is deemed 
to be essential to the cropped image and is thus prevented 
from being cropped out. 

In another embodiment of the invention, prior to 
30 identifying in the captured image any features, the 



electronic camera may be used t-r> 

5 face) itself, but in a diff ~ ~ ^ eXample ' 9 

out ln a different use model may also re ]^ P 
to appendages worn for- relate 

a *> worn for the purpose of identifier o 
example . „ J - uc «tirication. For 

-ade o' f u p : h r t s w r n9 a jacket ° f * 

- the ^ ; t thentheel ^ca. ratBybepinted 
« P-essU Ia ns ^ ^ * " ini — «- image 
-ociated w h a J^TT ^ ^ " ^ ~ 
is captured L an el ! ^ ^ ° f 

interest Z^ZZ ^ ^ ~* * « 
Particular a^Z^ ' 

15 the image processina mea ^ 13 CaptUred ' 

9 processing mea ns may then be used to identifv in 
the captured image at least nno . laentify In 

• • ne ta 99ed feature. Then t-h« 

compositional rules may be used ^ , 

used to crop the canfnr^ 
so that the tagged featm* • • captured i mage 

ayea reature is included in i-h^ 
image. Q ln the cropped 

20 

v) generate * —re of coition c^Uty for eac h 
alterative cantata crop boundary by L 1 a t . 
Picture oo^ sitlon ^ ■ Xy 



10 



- 8 - 



vi , selecting as output a reduced nu*er * crop 
candidates having a relatively high measure of 
quality, for exanple. Just one crop candidate. 

5 In an alternative e^odi-nt of the invention, step (Ui> 
comprises the steps of: 

rules as constraints 
vii) -7:^ le-r de.ree. each 

that ray r an irid - . 

10 constraint having an 

increases the less well the constraint is satrsfred. 

vii i> defining an overall cost metric as a function , of 
c j co-ordinates in the iaage. hy forming a co*>nat- 
15 Z the separate cost functions associated wrth each 
individual constraint; 

ix, applying an optimisation method to find one or more 
"It ZJ P locations hy finding mini™ in the overall cost 
20 metric; and 

x) selecting as output a reduced nu^er of crop candidate. 
1 said heat crop locations, for arable. Just one crop 



candidate . 



25 



30 



Th ere will be at least one crop edge, ror exa^le a circle 
TL ellipse has Just one crop edge. Often there «U he 
«. than one crop edge, for «*U . ~- ~ 
rectangle has four edges, between one and four 
rectangle n original electronic 

will result from cropping of the ongi 



image . 



pluralrty of crop edge*. the steps o£ ^ 

";rv eparate2y £or each — - ~ £ 

" eeTTT; This help to reduce the — »««- 

to select the crop tomdarles> „ ^ 
assessed independently from the ether edges. 

10 sir::, < whi : h the i ~ -* - - u 

regions of • ► °° nstltute "on-overlapping segmented 
f r t t " The " StSP *• — — by 

15 regl; of If T™"" " 

th! k lnC ° 0,086 " hiCh 1,111 »» ^-ded by 

he crop ^ dary and thoae wMch be V 

crop to—*. Bach-such aiterMtive division of regions ' 
Of interest is then need to determine en lnne r crop 
boundary l intt and . outer ^ ™P 

the er crop boundary l lmit ls the ^ 
which escribes the regions or interest to be indeed 
and the outer crop boundary limit is th. i 
-h eludes the regions^ in^stt ^ 
one or more suitable crop boundaries can ch en 1 
» determined between the iimits o £ the inner end ou crol 

r^orT ^ ° £ ~* 

regions of interest. 

, "hen the crop boundery has a pluraiity of edges 
30 - suiteble crop boundaries ere detUed^ ZZZ 
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, in for each possible edge placement 

- ~ r^r : z: « — * 

r; „ o£ ~ crop boundary -y at least in part be 
selection ot vest cj.^ f 
d epe„dent on the separata ado. quality .etrrcs o £ 

5 the boundary's edges. 

u -. ^ reduce the computational burden, as 
This can also help to reduce P 

- „ orflS sarv to consxder alternate 
lt is than not pessary c<OTpl etely 
„ he ra the outar crop bound n , l*t ^ ^ 

10 anclosa tna innar crop boundary 

o£ th a are. between the outar and inner P 
limits doaa not permit tha placement o£ 
boundary shape. 

v. in stop (i) blank or uninteresting areas 

15 It mt be that m step ( ^ ^ 

are detected as so.. o£ the £ ^ ^ 

coition and in steps Ux, nd Urr> 

ccw ositional ruies cause the -8. _ 

r-o the arrangement of saia 
according to the 

20 uninteresting areas relative c , 

relevant to the composition o£ the electrons image. 

u w.=^v areas are usually associated 
In photography, such blenK areas ^ 

•♦•v, a nlain background, or sxy. 
with a plain u a . ( . 4rtT1 . 1 rules associated 

«m use one or more compositional rules a 
25 means can use one extending 
v. v.i,„i, a rpas For example, a bianx 
with such blank areas. 

an uooer horizontal band of the scene may 
across an upper h appropriate 
associated with plain sky, and 

i ™le miaht be to minimise the amount 
compositional rule might . ontally the boundary 

• area and to orient horizontally 

30 particular area, a»v* 
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*n 9 ene ral . the - -teres, Th erefore. 

*• t*. arr a n 9a «, t of ^7 "\ CaptUred *-«• according 
5 — futures „ Wch _ * «~ r ela ti va to the 

If the in ®9e is a col 
°" e of performing t „e 

*nve„Uo„ is „ ^ ^ ^ "coercing to the 

«- — o image are - «- o^ sitiOT ot 

~ * seating the J f - - «e P UJ by . 

salient regions ^ J" tta others; 9roupin3 

-non. sali e M regW — «- oy 

9r ° UPS ° f redone as ^ — 

interest property features with a region of 

20 

Each such area of 

ux similar colour 

institute one of the ioentif ieT^es. ^ "* 

One way in which 

30 uivi^g the cepturea ^ * * «~ 

° 3 Polity of regions, 
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artt , /nr the texture within each 
averaging the colour and/or tne 

averaging grouped to 

region, ^acent ^^J?^^ having a 

low interest metric. Additionally ^ 

5 adiacent dissimilar regions may then be gr P 
5 ad 3 acenc interest having a 

delineate at least one type of area 

high interest metric. 

v me ,t-ric" is used herein to define 
The term "interest metric is 

y ....^i -mile For example a 

particular photographic conditional rale. 

feature with a high interest .netric *ay he assorted . th 
15 a rule that places such an area either at the centre^ 
cropped ina 9 e. or slightly to one srd of he cen 
i M ge. ft. e^le according tc the 
photographic compositional rule called 
thirds" . 

20 «f face detection, or 

xhe sy stem. may also he capable of *c 

person detection using one or more 

hair colour and approximate style 
recognition, hair coiou 

recognition, or use of body shape models. 

^ certain regions may be denoted as more 

^- v, alienee to regions on the basis 
by allocating high salience to y 

some combination of: 

n =o of the colour, intensity or texture 
30 - relative unusualness of the coiou 
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°f the region to other arf-i 

r ad3acent resins; and/ or 

" relative unu S ual„ ess of the «i 

°f the re9ion intensity or 

^ image. t0 * substantial portion of the 

A decision may then be 

- - u^ y the regio r: * and/or - 

*»• or how u ° U 3 «-*»•. face, head or 

» ~— or ho. ee„ tral *° '° * * -o™ 

re91 ° n ls in the ilage- 

^ SySte ™ «X ^so allow £or 

egion of interest. 

15 ^ positional rule for , 

— of the in » ge may fae ; o f 2l nt ; S t the 
cropped i^ge. e to delude such features in the 

invention will now b» „ 
20 only, with f be de scribed by W av of 

Y. with reference to i-h* 7 f Sample 

which: Che ^oompanying drawings, in 

1 is a blQck e 

^em incorporating an electro^ ^ ******* 
5 aCCOrdin ^ ^ the invention ha ^ Pr ° Cessin 9 device 

— pressor for :;^: a 7 ly a detector — - - 

-ages captured by the detector arra y ; 
Figures 2A to 2G a 

ulcaI1 y cropped; 
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j 3B are flow charts illustratxng an 
Figures 3A and 3B are in vention for 

embodiment of a method accords to 
cropping a captured electronic image; 

A 4B are flow charts showing in detail one 
Figur es 4A and 4B *™J fco the invention 

way of implementxng the method a 
as shown in Figures 3A and 3B; 

• 5 is a plot of crop penalty -trie against crop 
10 Figure 5 is a plot region of 

distance for alternative positions of 

interest ; 

Figure . shows statically minimum and — — 
15 rectangles for two identified features; 

, is a flow chart showing in detail one way of 
Figure 7 xs a flow invention as 

rtp method according to tne 
implementing the meth ^ „ const raint» based 

shown in Figures 3A and 3B, usxng 

20 approach; and 

ft is a flow chart showing in detail one way of 
Figure 8 xs a rxo invention as 

implementing the method — ~ ^ ^ 

shown in Figures 3A and 3B, usxng 
25 based approach. 

. , shows schematically an electronic camera 1 for 
Figure 1 shows Mniera ^ 

cspturing an electronic ^ the Bcene 2 

an optical imaging system 3 arranged ^ 
3 „ onto a two-dimensional detector array 



15 - 



25 



f " etMte array is connected 5 to a • 

£or pwessi ^ <* the iM9eo ° ; — . 

ar «y- The microprocessor , h P " ed " y **«*or 
The "^coprocessor 6 is lf - c lp 

°apture o„ tton 7 by wMch - - ima9e 

initiate capture of the in» Can,er * 1 «y 

— are varices UeTl ^ ^ '° ' ° " ** 

» conventional eiectronic 1^ a^"'" ^ ' 
«Wy. viewfi„de r or li„ T * ' ""'^ Power 

and Ught level ^JZTt- ^""^ 
- ~e ana auto-focus ™t c ^.— <«• 

^ture a faMhou ae 'll HFE— V" " . central 

'o the left o£ the e £ r at l eatUreS iDCl " d a: a dar t 
-tending across a " 3 h ° ri ~n line 13 

' above the hori^ l in f „ " """" * °* *— »' and 
foreground consists o£ ^J*™ 1 * ^ *r «. The 
featureless field a5 Th tUre " ""^ a nearly 

oataila u ia , lef Sre prions distracting 

*- one or more other ^ * *— » *»* 

~ C I- ^ — — or areas 
composition of the ^ are to the 

compositionally signif icant C lma9e ' h9Vin9 v ^°u 8 

^0.^ Properties z\:tt s ' por ~- 

^rmhouse U include its 
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central Nation in the captured — «*. restively 

7& level c £ detail chared with other features. - « 

x • , c- The sky 14 and field 15 share 
it a main area of interest. The sicy x* 

Monal property that these extend to borders of 
the composxtxonal property feat ureless. The 

5 the captured image, and are relatxvely 

horizon line 13 has the compositional property that x 
non-central, horizontal and well-defined in terms of 
Itrast. The details 1. have the compositional proper^ 
that they are not central to the captured xmage 10, «d 

,rea of interest to the maxn 
10 that they are a competxng area 

area of interest, i.e. the farmhouse 11. 

he formed for each of these 
Compositional rules can be formed 

composit ional propert ies . 

" According to conventional rules o £ photographic 

composition. the M » " «— £ 

particular, the m»in area o £ interest to the eye tb. 
£ arm»ouse XI. which is dwarfed by the expanse of sKy 
20 and £ ield IS. The extended horizon line 13 and detarls 
detract from the main subject o £ the farmhouse 11. 

Pigures 2 B- 2 G show one way in which the poorly composed 
Jge 10 can be automatically cropped to leave a much more 
2 S satisfactorily composed ima g e », as shown in Prgure * 
The first step is to process the image 10 a such a way 
that areas with similar colour, intensity and texture are 
„erged into areas having , uniform colour and intensrty. 
as shown in the image 20 in Figure 2B. 

30 
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a u areas is shown in Figure 3A f jT -c- 
itnaae m ^ , First, the captured 

space format. Areas „ lthi „ " "» col °« 

similar colour ana intensity ar I f ! ^ haVi " 9 

-witensity are then crenerat-^rf ~ 

^ first blurring the lma9e „ " r™^.™' ^ 

blurred inB9e - seed „ eas . 9 " in the 

15 not- « ^ » allocated to grown seed areas, if 

not 35, then the original YCC f nrm ^ • 

axudx ilc format images are blurred t- n 

yrown 33 as described above At = mo 

Point, the blurrino 3i of -u . At some 

"-tux xing 3i 0 f the ima ... 

such that all pixels an. *n sufficient 
20 " allocate d to grown seed areas 36. 

The processed image is then f„r+v, , 

adjacent areas separated b * -rging 

those separating , ^ 

separating areas having relatively i ow colouT . 

intensity differences. Next ar* a • " 

« colours are merged 38 ~ 1 

other laroer . ^ ar6aS enclos ^ by 

other larger areas are merged with the larger areas if ,J 
small areas fall Hoi lf the 

fall below a threshold size 39. when tM 
process is applied to t-h Q hls 
a mu.h • T captured image 10, the result is 

a much simplified image 20 h*,,,- 
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15 



30 



in Figure 2B. Each of these areas 21-25 has a uniform 
colour and intensity. 

The rest of the process for automatically cropping the 
image is shown in Figures 2C-2G and illustrated in the 
flow-chart of Figure 3B. First, an analysis grid 40 is 
superimposed 51 on the processed image 20 having the 
merged areas 21-25. The grid 40 defines a number of square 
regions 41 and in each of these the colour and intensity 
is averaged 52. The average colour and intensity within 
each region is compared with that for each adjacent 
averaged region 53,, and the differences are used to assign 
an "interest metric" to each region 41 depending on the 
difference between that region and adjacent regions 54. 
The various interest metrics are shown in Figure 2D as 
squares 45 of the same size as the regions 41 and having a 

i f« hhP difference in averaged 
brightness proportional to the oinere 

colour and intensity with adjacent regions 41. 

It should be noted that the description up to this point 
is just one example of how the initial processing of the 
image may be performed to identify, different features 
relevant to the composition of the electronic image. 

, E ach merged area 21-25 can then be classified 55 according 
to various criteria, for example its average colour and 
intensity, location in the captured image 10 and the 
various interest metrics 45 within each merged area 21-25. 
From this, it is possible to determine for each region one 
or more conditionally significant properties from 



- 19 - 



5 "-The farmhouse „ results " kSr " ^ the 

n results m an area with ►.(„>. • . 
"«rics 45. and the areas of a- 9 interest 

of the farl d "*»«*»9 «"*U 16 to the 

retries. ^ US * ^««e interest 

areas 21. 22 to k» u allows these 

•hich rein „ ^ ^ ' < ™ — » 

iocal maximum in the* 
15 separated from th* v. interest metrics 

— ,;^ t rir:ri metrics 45 - - 

street (ran the J a ~ » -«« «*. 

composition mies ca „ he .-L^T * 9 * in - 

competing areas of interest 2 3 that 1> 
20 cropped. " * n °" th ^e to be 



srmpie compositional rule, such as the -ml. , 
«n be associated with the main area of , IdS " 
farmhouse u. s dark „ „ ° f lnt «est, here the 

- - - of the riir * from - » - 

compositional rule that it • „ ass °<*ated with the 

awa y fro m a main ar ea of T ^ * "~ ^ «- 

- croPPed ^:z:iz Bre8tas a naturai b — - 

appropriate to each t ""^"""A **~ 

™ , type of mer 3ed area 21 ,c 

30 selected 56. This i<* oh . 5 can be 

mis is shown in Figure 2P 

-Hjure 2B, where a three-by- 
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three rectangular grid 50 is superimposed over the grid of 
areas of interest 45 in such a way that the left-hand edge 
47 coincides with the location of the dark merged area 25 
associated with the dark tree 12, and with a one-third 
5 line 42 running vertically through the highest interest 
metrics 45- The rectangular three -by -three grid 50 is then 
positioned vertically so that a highest interest metric 45 
is centred vertically in the grid 50. Therefore, by 
applying the selected compositional rules, a suitable crop 
10 boundary can be determined 57 . The system may then present 
this crop boundary to a user of the system as a suggested 
crop boundary. 

Figure 2F shows a captured image 10 with the rectangular 
15 three-by-three grid 50 positioned as determined according 
to the procedure described above. The captured image 10 
can then be cropped along the outside boundaries of the 
rectangular grid 50 to result in an image 59 cropped 
according to the selected compositional rules. It will be 
20 apparent that the cropped image 59 has a better 
composition than the original captured image 10. 

Preferably, the processing of the captured image 10 is 
performed by the microprocessor 6 in the camera 1, with 

25 the memory 8 serving to hold data generated by the 
automatic cropping process, and the resultant cropped 
images 59. It would, however, be possible for image 
processing to be done external to a camera body, in which 
case the electronic camera and external processing form 

30 the electronic camera system of the invention. 
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The invention is particularly uaeful in the oase o£ 
electronic cameras having a detector array 4 with , 
relatively large number of detector elements. For e*an*le. 
a detector array having two million or m elelnents ^ 
be used with an optical system 3 having a wide ang!e field 
Of view. The nser then need only point the electronic 
camera 1 generally in the direction of a scene 2 he wishes 
to photograph, automatic cropping can then he nsed as 
10 described above to crop unwanted areas of the captured 
«age. This relieves a considerable burden fro™ the 
Photographer, as he no longer needs to w,rry unduly about 
detaUs of photographic composition. Electronic 
Photographs can then be taken rapidly, which increases the 
chances that the photographer will capCure che desired 
moment. 



One way of manually tagging futures is to use the camera 

vrewfinder as a pointing device ,as opposed to its 

conven .onal use as both a pointing and co^osition 

device,. In this use ^ ^ _ ^ 

deliberately positioned in the approximate centre of the 
frame. When the image is auto-cropped according to this 
invention, the region at the centre of the image is deemed 
to be essential to the cropped image and is thus prevented 
rrom being cropped out. 

in another variant of the invention, the ca TO ra may be 

initialised to identify cer h an n „ i 
30 • . certain colours or texture as 

30 having a high interest *t- i 

ncerest. At least two use models are 
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possible here. One simply involves the identification of 
features of "natural" importance or inherent interest: 
faces, the overall shape of a person or object, and other 
expected compositional elements. Another is to provide 
additional elements or appendages for the specific purpose 
of "tagging" to force inclusion of an object in the 
cropped image. The practical effect may be similar in 
either case. For example, if a person is wearing a blue 
rain jacket, then the camera may be pointed close up at 
the blue rain jacket and then capture an image of the 
jacket. The camera can then be programmed to process a 
captured image to assign that particular colour a high 
interest metric. If a wide angle picture is then taken of 
a scene in which the blue jacket appears, then this area 
can be assigned the highest interest metric so that the 
captured image is automatically cropped in such a way that 
the blue jacket is retained in the image. This is 
particularly useful when images are captured of a crowd of 
people, one of which the photographer would like to make 
the main subject of the photograph. 

Figures 4A, 4B and 5 show in detail possible steps for 
implementing the steps 55, 56 and 57 briefly outlined 
above in Figures 3A and 3B. 

Once features relevant to the composition of the 
electronic image have been identified 60, as the steps up 
to and including step 54, a minimal cropping rectangle, 
(MCR) , and associated cropping limits can be defined 61 as 
crude limits on the minimum and maximum areas of a cropped 
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image. If the interest map f or an lmage genetated in 60 
shows N distinct areas of interest. < £ „ r exanple. areas of 
interest separated by some area of non-interest determined 
by some adaptively set threshold), possible minimal 
cropping rectangles can be generated which contain 
alternative combinations of between 1 and » areas of 
interest .here the MCR contains the selected combination 
°f areas of interest and excludes the other areas. Mote 
that not all combinations may be possible, as these may 
not be contained within a single rectangle that excludes 
the non-selected areas. The cropping limit for a given MCR 
is the maximum rectangle, which contains the MCR but 
excludes the non-selected areas of interest. This is shown 
xn Figure 6. which shows Jor . capture<J 9o ^ 

dxfferent minimum cropping rectangles 84.85. that win be 
explained in more detail below. 

Each MCR 84 or 85 and its associated cropping limit is 
processed in turn. The first step ^ ^ # f ^ 

or 85 as a current potential cropping candidate 62 
Steps 63 to 7 „ essentially set up a set o, preferred 
constraints on possible crop locations and then 
successively generate candidate crops that meet some 
combination of these constraints, m practice there win 
» typrcaliy be too many constraints to satisfy all 
constraints simultaneously, step 63 in particular in 
»hi=h a set of possible crop edge iocations is identified 
for each location as edge constraints, deliberately 
suggests multiple possible iocations for each edge 
30 However, only one possibility can be used in any one crop ' 
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Steps 64 and 65 are given as examples of the 
implementation of the compositional "rule of thirds" as 
applied to the identified features of horizon line 64 and 
a main area of interest 65. 

The idea behind Figures 4A and 4B is that all combinations 
of position are generated and then evaluated in the later 
steps. This is a "generate and test" approach to 
determining one or more suitable crop boundaries, using a 
"constraint set" determined by minimum and maximum 
cropping rectangles . 

Note that step 66, where a question is asked as to whether 
or not the cropped image is to have a fixed aspect ratio, 
ensures that it is possible that a particular aspect ratio 
can be specified and enforced. In practice this means that 
when an aspect ratio is specified 69, a smaller number of 
other constraints will in general be required 67 to 
completely specify a crop candidate, than in the 
alternative case 68 when no aspect ratio is required 70. 
In the case of no explicit aspect ratio requirement, it is 
likely that an evaluation rule in step 10 will penalize 
ridiculously thin aspect ratios. 

Once a crop candidate has been identified, it is then 
evaluated 71 by one of more rules. Each rule is 
implemented as a heuristically evaluated measure on the 
image. For example, a metric 72 which measures how close a 
point of interest is from a one-third line is shown in 
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IZ l , £raCti ° nal P ° Siti0n ° f "» »*>* *» the 

candrda e crop is measured in both the horizontal ^ 

vertrcal direction.. The penalty £or each direction 
aete^ned from the heuristically determined g ra P h shown 
Figure 5. The two measures pe „ penalty 
are combined by the rule: 

penalty . max< penal ty„„ , penal t y tol , , 
if max( penal ty„„ , penalty,^ ,>«,.„ 



and 



20 



25 



30 



penalty = mean( penalty^ , penalty^ , 
max( penal ty vert , penalty^ ) s 0 .75 

^ilar heuristic measures _ ^ 

corona! ^ such as eliminating distractions close 

to the edge of the f rame , minimum edge 

preference for dar k or low activity ^ ^ ^ 

The combination of different- 

sum £enmt P^alties by a weighted 

sum allows some rul^c? K ^ 3 

t-h. v ° bG con ^dered as more important 

FoTLaTl POSSibU ~ 0nS ° f tte -heme. 

For example. « ^ be possibie tor the rule combination 
waxght ngs to be dynamically adjusted accord^ to the 
-ran type o£ image Fm a ^ the 

area o f interest containin g a si„ 9 le faC e loo k n 
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straight towards the camera may reduce the weighting for 
the rule of thirds, allowing a more centrally placed 
portrait to be preferred. 

S Another possibility is for an additional penalty factor to 
be generated from step 61 where some MCRs are 
intrinsically preferred (i.e. given a low penalty) 
compared to others. 

10 In Figure 4B, the penalty is evaluated as follows. First, 
a test 73 is performed as to whether or not the total crop 
penalty is less than a lowest previous total crop penalty. 
If so 74, then the current crop candidate is recorded 75 
as the best crop candidate so far. If not 76, then a test 

15 77 is performed as to whether or not there are more 
constraint combinations left to test. If so 78, then the 
flowchart loop back to step 66. 

If not 79, the flow chart next tests 80 if there are other 
20 minimal cropping rectangles 84 or 85 left to test. If so 
81, then the flow chart loops back to step 63. If not 82, 
the flow chart shows that the best crop candidate is 
returned 83 as an output from the process. 

25 Figures 7 and 8 show in more general terms the concepts 
behind respectively the -constraint -based" and "generate 
and test" approaches described above. In both cases, the 
starting point is the identification of features relevant 
to the composition of the image 91,191, for example as set 

30 out in the steps up to an including step 54 in Figures 3A 
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and 3B. 



25 



30 



S ZL' , en " fied »- —pt of a 

3 constraint set" is a * 

u- u set of alternatives, only one of 

"hich should be considered at a n« » v 

at a tlme - Each alternative 
consists of one or specific ernatrve 

6Mmnl _ Xy s P eci fied constraints, for 

exa^pla £eatures „ ith ^ . 

enumerated 93. en 

0 

^7*," a oonstraint set is 

AUaLt aUernatiTC - »a -Unascapa-. 

Alternative 1 (mrfr a -;f\ - , 

m „ . (Portrait) might be def . 

constraint : y cne 

Wight - L e f t ; / (Bottom . Top) = Q ?5 
Alternative 2 (landscape) might ^ 

constraint: y e 

Mght - Left) I (Bottom - Top) .1.33 

-ould dafana c r op pi „ 9 limit alternatives 
groups of areae o£ . nt "'rant 
boundary 90 of ,>,.•■, »itlun the maximum 

IfJT 91 " CaPtUred iM9e - 1,16 -*« of 

ha a ^ " " S " 13 * «- of 

the areas of lnterest .„ ^ 

of xntaraat M . es have haen . aatermined with pixel 
co-o-Waa.- ' - ««0. 300, aud ,3*0.50^ 
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(6 00. 2 40, as shown beiow. In this example, the whole image 
has a top left co-ordinate (0,0) and bottom rrght 
co-ordinate (640,480). 

6 The orop limits constraint set would consist of three 
alternatives: 

Alternative 1 (left-hand area of interest 84 only) is 
defined by the constraints: 

10 

Left > 0 Left < 30 

Top > 0 Top < 180 

Right > 200 Right < 350 
Bottom. > 300 Bottom < 480 

Alternative 2 (right-hand area of interest 85 only) is 
defined by the constraints: 

Left > 200 Left < 350 

20 Top > 0 Top < 50 

Right > 600 Right < 640 

Bottom > 240 Bottom < 480 

Alternative 3 (include both areas of interest 86) is 
25 defined by the constraints: 

Left > 0 Left < 30 

Top > 0 Top < 50 

Right > 600 Right < 640 

30 Bottom > 300 Bottom < 480 



- 29 - 



constraint set concept can be used to 
-t«Uy exclusive sets of altem t 

include: aspect ratio , alteni »»«. Typical examples 

-ous 9 roup in9 s o* orop ^ ° £ 

alternatives (bottcn third li„ " plac ^' 

- — placement 2 'ZZ^T ^ ' ~ 
intersection points or , , "* '° ur '""-third' 

» the tour -one-tnird- Un, " . ^ ~ * 

«- -p. .otto™. l8£t and^ h ; d X £erred P — s 

For each edge there .<= 

^native ra^es o f KsLT^Z " « 
» hasis of an ad ge w i ity «« - the 

The examples given above are all -h, A. 

the condition »st be J J^h C ° nStrai " tS - ^ 
P-alty involved in deviatino f "° 9radUal 

*° cases it is desirable 2 , C ° ndi "° n ' Ia -V 

that ls , incurrJ g „ '™nt S as -soft-. 

- solution ^r^T^ ^ ^ 

that positioning a horison linTLTtlT' ^ ~ *" 
"ne is better i^ lenented Jn / - the one-third 

25 a little way off t h. a " OWS P^ement 

» en the precise one-thirj 
Penal ises increasing t J\^ *°*^> "« 

location. ° m the de sired one-third 

The optimisation problem can 
30 thiS - — »« >V chan ging the Citron;" ^ 
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x = 1/3 

to 

x + el - e2 = 1/3 

where el and e2 are positive penalty terras which 
contribute to an overall penalty function to be optimised, 
typically as a weighted sum of contributions such as... 

penalty = del + c2e2 + 

The next step is to pick 94 the next combination of 
constraints, one alternative from each constraint set. 
Many combinations can be immediately excluded or 
simplified as they are precluded by some of the other 
constraints. For example, choice of a particular cropping 
limits alternative will limit which points of interest can 
be considered as some may be outside those cropping 
limits. 

The optimal crop candidate for the current constraints can 
then be determined 95. The constraints we have set up are 
combinations of simple linear conditions. These can be 
effectively solved by linear programming methods which 
find the location for top, bottom, left and right 
boundaries of the crop which meet all the hard constraints 
and satisfy the soft constraints in such a way that the 
overall penalty is minimised. 

Depending on the precise set of constraint combination 
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being solved, there may be a 

— to . — :,° £ si — 

Place A near the riot! f COTStralnts «« attests to 

cne right-hand one- third Hr,~ 
left-hand one-third line t L t * °» 

10 ~ - method in oT /' "° 

constraints to solve shouid -^11 b ^ «* 

eliminate these situations. e "° U9h te 

There „a y „. ^ 

15 score. i„ thls ca equally i 0 „ penalty 

» case we have a number of alt.™.* 
ls to pick a solution at „ alternatives. One 

-"Pie solutions. M he r " T ^ •** «~ - * 
«- «a,le b y t „min 9 Z * «~- - 

haro constraints. Optionally, in^ep 96 t^ ^ 
2 ° « a richer evaluation I trio to " P ° SSlUe * 

alternatives » lthin the " a set of 

solutions and select «.= equally acceptable 

evaluation TOtric " ^ °" basi * « the refined 

a -generate and te« " *T "~ *" ™' - 

» Possible. meth ° d - ^ Of this are 

The linear solution i<= = 

- - autct jxl™ wm - 

formed that represent *.* cons tra ln t sets can be 
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generating <3°o« alternative, o ^ 
optimisation methods P £or globa lly 

optimal solutrons. » cunvent such problems in 

space allows this technique to circ 
a relatively intelligent manner. 

,™ is one method that may be used 
though linear programing <^ „ ay the 

in step ». it does rmpose techniques 

v defined. Other optimisation 
oonstrarnts are derme ^ ^ 

rii:": — - - ~ - 

constraints from constraint sets. 

; I£ all possible constraints and -^.^^ 
encoded as hard or soft conditrons ~» P 

in step ». then step X may be W-"*^ _ q£ the 

thieved by linear programming then 
95 is achieved ny omitted. A more 

k~ nnorlv approximated or omitce 
0 constraints may be poorly app enprated by st ep 95 

, nf the solution generated oy 

accurat e ^ 3tep „. * „>re reiined 

can be obtained after* te s o lu tion from 

i^iementatron mrght use t J tesed 
6tep 95 as the start pornt for a gen 
25 local optimisation usrng the more 



metrics . 
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• - t-hat can be only approximated 
M exan^le of constrain s that can ^ 

„ ith a linear representation- « J- ^ ^ ^ 

The true edge quality can only really 
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limits of the edge are 1™ 

*— t on tne top Md ^ locatlons is 

<* ^ and ^ TOy » narrower choice 

5 would otherwise advereelv „„ ^ that 

"near eyate™. The be3t t . ? ° an "° t 1)6 with a 

«-t.. the edge n1 "*" ^ 

»• for exa^e. the ^ " """^ uaino. 
specif ae IT aUer t Cr ° PPin9 * 
constraint set. ^ ^ II ~ *"* *" ^ 
«- Particular choice „f ™ — «hiie 

considered. CrOPPin3 is being 

5 

Mother type of conatraint that , 

ia on. W.in, ratJa £ ^ ^ 
for exa^u, the uUt)n ar «a r„ the lm a 9e , 

the crop boundary, clearlv ».■ 9 re9l ° n Within 

- - a ^tipTjir ; r : r "°"- anear - 

a"" vertical crop iocationa. 1V ' n9 * 

In any event, once> t-h~ 

«- is recorded", a ^ ""^^ -luated. 

' alon9 uith its penalty SCO re. 

a teat ,e is perforaed „ 
constraint aet co*inatio„a have been e , "* *" 

«- flowchart loops hach to atep „ If " "* 

"any posaibilitiaa for ' S °' then there a « 

-Ite. K igU re 7 sho „: "I ^ ^ * ' * 

3oat one example. i„ „ hich ^ 
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, ^rrier of penalty score 101, 

- — - °z: n i — — » 

then a number or 
and then ^ ^ system . 

scores are presented 102 to a 

a set of alternatives, a 
m addition to selecting from a set 

• C h to suqgest improvements to 
user may wxsh to sugg gimpi commands 

alt emative. This could be achie ^ 

^ nipqc" In such cases t"<= ^ 
such as or less . ^ crop 

define a ne» mnxmum crop rectang alterIl atives 
oesed on the seated crop and the ^ _ 

requested -more" then the MleC " non _ sel ected crop 

which exceeds the size of the re . cons ider 

i-imit- The system can then ie 
new maximum crop lxmxt. Y fcQ 

edqe placements within these 
alternative edge p t to the user. 

Repeated use of this form alte rnatives 
to interactively explore ^ the spa ^ 
0 which best meet the system's criteria 
acceptable crops. 

le of another approach would be to ensure that one 
An example of anotner w limits was 

from each of the possible alternative crop 



25 used. 



lns0m e applications o. the invention, a user » y not he 
involved, and the crop « be fully autorcatxc. 
30 P ig ure S is a 9 eneral e^le of a generate and test- 



35 



5 



10 



method, in which certain 

—i«y constraint - as nard. 
candidate witnout an y att "T ; SSd t0 de " ne 3 

—taint is a loca, °P"»isation. Each tard 

criterion. l0Ca "» «« a particular 

yure 7 are qiven a ^ c 

incremented by ioo. Terence numeral 



Once the relevant constraint sets for the , 

l>een determined 192 , these ^ the feat ^es have 

grouo r>f 6 enum erated i 93A 3nH 

» — o f c 0nstrs :rr s ;r; sets a ~ such ~ — 

A sil «ple example i s f or 

« « «*t locations , where c ; et ; £ -p. ^ laft 

candidate edge position det • constraints is a 

metric. " to- the edge ^lity 

S " ^ - constraint set might 



Left = 5 
Left =38 
Left = 150 



30 Analogous constraint sets would exist for ■ 

exist for possible Right, 



- 36 - 



Top and Bottom candidates. 

ahnve the driving constraints 
in the example discussed above, cne a 

are edge constraints such as these, coined with the 
5 cropping limits for various combinations of areas of 
Interest (i.e. minimum crop rectangles and cropping 
limits) • 

» alterative group of driving constraint sets might be 
1. three enumerated edge location constraint sets and an 
aspect ratio constraint set. 

The driving constraint sets determine which out of all 
possible crop rectangles are -generated-. In the least 
» intelligent possible Mentation, all possxble left 
right, top and bottom, locations are generated. This, 
however, increases computational effort. 

m step 194, the next combination of driving constraints 
20 is selected, one from each driving constraint set. 

The determination of the current crop candidate in step 
195 is trivial, as there is only one possible solution to 
' the driving constraints by definition. 

" in step 1«, all soft constraints are evaluated. Here, an 
unspecified collection of soft evaluation criteria are 
combined . 

30 once the crop candidate has b.en evaluated, this is 
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= fT «. - a^t: rz* "z t testea - - 

flowchart only i oonc t0 Ste P 1M. The 

7 lo °P s round the driv,„„ 
these fully specify ^ ^ d " Vln 9 constraints as 

considered. —angles to be 

° If a11 living constraint u 

«"» "suit,. P igure . " *~ -t to do with 

-op c,„ dldates are ~ 3USt « —-J.. « „ hioh the 

- - , niraber err soore 2ci - 

-res are „ ed 202 to a _ ^ 
The invention provides va w 

improved selection and m . Cro PP ed images with 

— - nnr - 

Photographer to take nhr* P3rt of the 

^ t: 9ood ana ~~ 

— CMKra to Che of an 

quickly. 8 lar 9 e n umber of lmages 
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Claims 



10 



r An electronic image processing device for cropping an 
!. An electro processing means, 

electronic image, comprising an rmage P 
the image processing means including an 

l::: a n; «— -/or - - 

electronic image, .herein the device xs adapted to. 

a) process the electronic image to ""^^ 
reatures relevant to the citron ofj- ^ 
iMge . each such feature occupying a suh reg 
electronic image; 

b) select one or more ^ 
15 P^erminedco^t^l-es.^^ ^ ^ ^ 

of the compositional rule is) 
identified features; and 

c) aetermine one or more suitahle crop houndarie= > by 
applying one or more of the selected compositional rules. 

2 „ electronic camera system for capturing ^ an 

■ • „™ of a scene, comprising, a detector array 
electronic image of a P 

£or capturing the ~ = ^ " J t „ e deteoto r 
sy stem arranged to .mage the scene ^ 
array and an image processing device 
I^ed image, wherein the im»ge processing means is 

claimed in Claim 1. 
30 3. An electronic camera as claimed in Claim 2. in which 
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the camera includes means by which . 

may tag one or more Y ^ h * of the camera 

«- electronic * C ~ i0n 

delated with . 4^ Tli^r^ 
tagged feature (s) in t-h includes said 

tS) ln the cropped image. 

4 * A method of usinr, 

P-cessi,* toto ' c ^- «- i. 3 e 

- — Processing p :i M in ^- . — . 

Processor and f irnMare "^sg as electronic 

— c ^* '« tne 

4-W esch such featu ! ° mP ° 3ltl ° n ° £ *»■ iconic 
electronic image; CUP5 ' in9 3 -*"«91« of the 

- «- co nPO s itiOTal ^(s, ;t„T o : therelevMre 

testified features; and m ° r ' ° f °» 

25 iii) determine one ™- 

applying one or more of t r T^' ^ * 

of the selected compositional rules. 

5 - A method as C lai med in Q . 
identified feature has one or " "«* 

3 ° significant properties frora *** COn *> osi tionally 

from amongst a plurality of 
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different predetermined compositional properties, and one 
or more compositional rules are selected from a set of 
predetermined compositional rules, based on the relevance 
of the compositional rule(s) to the compositionally 
significant properties of one or more of the identified 
features . 

6. A method as claimed in Claim 4 or Claim 5, in which 
step iii) comprises the steps of: 

iv) generating a plurality of alternative candidate crop 
boundaries; 

v) generating a measure of composition quality for each 
alternative candidate crop boundary by using a set of 
picture composition metrics; and 

vi) selecting as output a reduced number of crop 
candidates having a relatively high measure of composition 
quality. 

7 . A method as in claimed in Claim 4 or Claim 5 , in 
which step (iii) comprises the steps of: 

vii) implementing the compositional rules as constraints 
that may be satisfied to a greater or lesser degree, each 
constraint having an associated cost function that 
increases the less well the constraint is satisfied. 

viii) defining an overall cost metric as a function of 
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crop co-crates i n the image , ^ fo ^ ^ 
individual constraint; 

i::j;: n r tions * — - - —» -* 



ior said best crop locations. 

Inly one^ " ^ * ^ < « ^ which 

only one crop candidate is selected. 

15 9_ A m ethod as cla i me d in any of Claims 4 to 8, in which 
he crop boundary of the cropped ima 9 e has a plurality ^ 
-op e_d 3 es, and the steps of ^ ^ J ^ 
separately f or each crop edqe to „ 

boundary. 9 t0 the C ™P 
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10 - A method as claims ,„ 

Which: " ^ ° f CUlmS 4 to 9, i„ 



- step ,i, inoludes identifyin9 features 

25 non-overlapping segmented regions of interest; 

- step (Hi) is achieved by fir3t selec alterna 
divisions of said identified reoion, f <=«™tive 

„Mch „Ui he incl J ^7 crop TT ^ 
30 those which .in be excluded J e :;~ d - - 
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- each such alternative division of regions of interest is 
used to determine an inner crop boundary limit and an 
outer crop boundary limit such that the inner crop 
boundary limit is the smallest boundary which 
circumscribes the regions of interest to be included and 
the outer crop boundary limit is the largest boundary 
which excludes the regions of interest to be excluded; and 

- said one or more suitable crop boundaries are determined 
between the limits of the inner and outer crop boundary 
limits for each of said alternative divisions of regions 
of interest. 

11. A method as in claimed in Claim 10 in which said 
boundary has a plurality of edges and one or more suitable 
crop boundaries are determined by evaluating an edge 
quality metric for each possible edge placement between 
the inner and outer crop boundary limits, the selection of 
best crop boundary being at least in part dependent on the 
separate edge quality metrics of each of the boundary's 
edges . 

12. A method as claimed in any of Claims 4 to 11, in 
which one or more of said suitable crop boundaries are 
presented to a user of the device for manual selection by 
the user for cropping of the electronic image. 

13. A method as claimed any Claims 4 to 12, in which in 
step (i) blank or uninteresting areas are detected as some 
of the features relevant to the composition and in steps 
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30 



(ii) and (iii) one 

cause Mw • compositional rules 

of said b T to be cropped according to the «— 

l^LT " ******* «~ dative to other 
identified features relevant to 
5 electric i^ge. «*«tl- of the 

«. A TOthod as olaimed ^ ^ of 

o LxTT ed by swi °* the «- ™ 

similar colour and/or texture. 

«. A method as claimed in any of claims 4 to 14 in 
which i„ st ep (i) features relevant to «. 
hh<= ol . relevant to the composition of 

the eiectronic image are i^tified by . 

- segmenting the image into regions; 

- denoting certain regions as more salient than others.- 

- grouping salient regions into 1= 

by relative „ , ^ rS9i0nS grated 

oy relatively non-salient regions; 

- identify said groups Qf saiienfc ^ 
with a region of interest property. 

16. A method as claimed in claim is u- u 

is *<™> ^ • ' " whlch the image 

segmented into ^ninne u 

into regions by segmenting the imaoe 

"Ire COmbina " 0n " «*~' "V - 
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17. A method as claimed in Claim 15, in which certain 
regions are denoted as more salient than others by 
allocating high salience to regions on the basis of some 

5 combination of: 

- relative unusualness of the colour, intensity or texture 
of the region to other adjacent regions; and/or 

10 - relative unusualness of the colour, intensity or texture 
of the region relative to a substantial portion of the 
image. 

18. A method as claimed in any of Claims 4 to 17, in 
15 which the compositional rule for features identified at 

the centre of the image is to include such features in the 
cropped image. 

19. A method as claimed in any of claims 4 to 17, further 
20 comprising a step (0) prior to step (i) of determining, 

independently of capture of the electronic image, specific 
features or objects of interest, and wherein step (i) 
further comprises attempted identification of said 
specific features or objects of interest and designation 
25 of successfully identified specific features or objects of 
interest as features relevant to the composition of the 
electronic image. 

20. A method as claimed in claim 19, wherein step (0) 
30 comprises identification of specific features or objects 
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prior to capture of the electronic image. 



21 



A method as claimed in cUi „ 19 _ where . n 
comprises provision o £ one or more identifier tags and of 
rules for identif ioation of identifier ta 9 s. and .herein 
«ep (1 , comprises use o£ Mid for idenci£ . cat . on 

said identifier tags. 

10 xdentifxer tags are infra-red transmitters. 

»• An electronic imaging processing device for cropping 
» electronic image, substantia!^ as herein described 
with reference to or a«? 
15 drawings. ™ ^ 



». An electronic camera system for capturing an 
electronic image of a scene, substantially as herein 
described. „ ith reference to or as shown in the 
^0 accompanying drawings. 

Lc A r thod ot usins an eiectroni ° ^ 

dev.ee for cropping an electronic image, substantially as 
herem described. w ith refer _ to or ^ ^ ^ 
^ accompanyang drawings. 
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